Focusing Attention with Deictic Gestures and Linguistic Expressions
نویسندگان
چکیده
Comprehension and production of text and discourse do not solely depend on linguistic expressions, but also on the physical context. The questions addressed in this study are 1) whether deictic gestures are substitutable for deictic expressions, and 2) whether deictic gestures establish joint attention. An eye tracking experiment investigated the effect of referring expressions and gestures on various aspects of attention. Results indicated that deictic gestures substitute for location descriptions. Furthermore, the manipulation of the synchrony between gesture and speech showed that hearers benefit from focusing of visual attention.
منابع مشابه
Deictic Object Reference in Task-Oriented Dialogue
This chapter presents a collaborative approach towards a detailed understanding of the usage of pointing gestures accompanying referring expressions. This effort is undertaken in the context of human-machine interaction integrating empirical studies, theory of grammar and logics, and simulation techniques. In particular, we take steps to classify the role of pointing in deictic expressions and ...
متن کاملAutomatic Referent Resolution of Deictic and Anaphoric Expressions
Deictic and anaphoric expressions frequently cause problems for natural language analysis. In this paper we present a single model that accounts for referent resolution of deictic and anaphoric expressions in a research prototype of a multimodal user interface called EDWARD. The linguistic expressions are keyed in by a user and are possibly accompanied by pointing gestures. The proposed model f...
متن کاملA New Model for Generating Multimodal Referring Expressions
We present a new algorithm for the generation of multimodal referring expressions (combining language and deictic gestures).1 The approach differs from earlier work in that we allow for various gradations of preciseness in pointing, ranging from unambiguous to vague pointing gestures. The model predicts that linguistic properties realized in the generated expression are co-dependent on the kind...
متن کاملCombining Deictic Gestures and Natural Language for Referent Identification
In virtually all current natural-language dialog systems, users can only refer to objects by using linguistic descriptions. However, in human face-to-face conversation, participants fre= quently use various sorts of deictic gestures as well. In this paper, we will present the referent identification component of X T R A , a system for a natural-language access to expert systems. X T R A allows ...
متن کاملTowards a Cognitively Based Approach of a Description of Spatial Deixis
In this presentation an approach towards a description of spatial deixis based on the perceptual and cognitive abilities of humans is outlined. After a basic introduction into space perception and representation the ndings of this part are taken to form the basis for a characterization of the phenomenon of deixis as well as the conceptual components of deictic expressions in a natural language....
متن کامل